Learning Objectives

By the end of this session, you will be able to:

1. Distinguish primary vs. secondary data sources and when to use each
2. Design effective questionnaires and survey instruments
3. Select appropriate data collection methods for your research question
4. Apply observation techniques to a real-world scenario

Artifact Output: What You Leave With

Types of Research Data Collection (Source)

• Primary Data Source
  • 1st hand information
  • not changed by any individual
  • not published yet
  • directly collected by authors
• Secondary Source
  • published data
  • what literature review is based on
  • reviewed by authors(you)

Examples of data collection methods by category

• Primary Data Collection Examples
  • Questionnaires
  • Interviews
  • Focus Group Interviews
  • Observation
    • participant/non-participant
    • aware/non-aware
  • Survey
  • Statistical Methods
  • Experimental Methods
• Secondary Data Collection Examples
  • Published Papers/Sources
  • Databases
  • Books
  • General websites
  • Unpublished personal records
  • Census data/population statistics

Primary Data Collection Methods: Designing Questionnaires

• a set of questions and secure answers from respondents
• often analyzed by statistical methods
• consistency in questionnaires make cross-sectional analysis easy

Types of questions designed to measure variables in survey:

• Close-end questions
  1. Two-option aka dichotomous scales
  2. More than two options: Nominal-polychromous
  3. Ordinal-Polytomous scale
  4. Continuous or bounded types
• Open-end questions
  • sentence completion
  • open-ended questions with free text responses

Polytomous Variables, aka Options for Multiple Choice Questions

Statistical term that refers to a categorical variable with more than two possible categories or levels

Common in: Survey research ,healthcare, market research and education

Characteristics:

• Categorical data: limited/distinct values or categories that are mutually exclusive
• Always more than two options(binary)
• Ordered and unordered (natural hierarchy)
• Statistical analyses: e.g. logistic regression, cluster analysis for patterns and trends
• Convertable to binary variables

Purpose:

• Understanding complex phenomena: patterns and trends in the data collected
• Provide insights to better categorization of data
• Statistical analyses: Relationships between variables: e.g. logistic regression to predict probability
• Market segmentation: e.g. customers to preference, behavior, demographic groups

Polytomous Variable (Continued)

When to use Polytomous Variables

• Measuring attitudes or perceptions: for more nuanced perceptions
• Putting definitions on categorizzing data: greater variability
• Analyzing relationships between variables: job satisfication & job performance
• Need to capture complex phenomena

Limitations

• Power of explanation limited by available options
  • e.g. satisfactory survey & number of enrolled students, what about change?
• subjectivity from questionnaire design and interpretation of results
• response bias
• small sample sizes
  • calculating the appropriate size of population needed
  • amount of respondents needed to reach statistical significance (reject null hypothesis, not covered by current class, leverage sample size calculator online)

Primary Data Collection: Designing Questionnaires

Face-to-face, paper-and-pencil or remote, make sure your data collected is: - well-organized and - easily accessible for analysis.

General rules for constructing a questionnaire:

Dos:

• questions should be short and simple
• provide clear navigation to avoid difficulty in reading and motivate answering
• use positive sentences
• add open-answer possibility after provideing listed answers
• improve reliability by selecting appropriate words
• explain importance of the questionnaire
• order your questions to solicit the right answers (sensitive to follow concrete/innocuous ones)

Do-nots:

• use more than one question (double-barreled) in one item
• make assumptions for the respondents
• lead the respondent to answers with clues, suggestions and hints

Steps involved in designing a questionnaire

Primary Data Collection: Interviews

Face-to-face and remote (telephone/zoom) interviews and merit/demerits (Kabir, 2016).

Good for complex or sensitive concepts and need detailed and high-status information (Frechtling, 2020).

Types of interviews by structure:

• structured interviews: standardized questions that are pre-prepared
• semi-structured interviews: conducted based on guide but goes beyond list of questions
• unstructured interviews: informal, casual conversations

Rundown of an interview process

Primary Data Collection: Focus Group Discussion (FGD)

• Mixture of interview and observation
• Used to discover human behavior, attitudes and respondents facing a particular concept
• 6-12 people in each group with shared characteristics
• Mediator aims to stimulate and discover the behavior of the participats and reasons for each behavior using the social dynamic of the group

FGD: Strengths and Weaknesses

Strengths:

• discover social, health and cultural concepts
• literacy of individuals non-issue
• suitable to explore complex subjects
• useful to develop hypotheses

Weaknesses:

• expensive and time-consuming
• privacy risks
• confined by readiness of facilitator/mediator
• domination of limited individuals in focus group (Frechtling, 2002, Kabir, 2016)

Survey and questionnaire: What’s the difference?

• Questionnaire is the written set of questions.
• Survey is both the set of questions and the process of collecting, aggregating and analyzing the responses from those questions.

Survey: Example survey accompanying sheet

Good & Bad Survey Questions: Let’s try out how to conduct surveys

Context: Surveying respondents on their religious beliefs and life styles

1. How religious are you?

• vague, not sure what is being asked
• How would you rate your level of spirituality? Pick a value between 0 to 4

2. What do you think about smoking on campus?

• vague, too many possible answers
• Do you believe that all buildings on campus should be designated as smoke-free?

3. How important is spirituality in your life? Pick a value between 1(not at all) to 5 (Very much so), (3: somewhat important).

• leading, suggests that spirituality is important
• What role does spirituality play in your life? Pick between 1(not important) to 5(very important) with 3 being somewhat important

Recap: Tips for effective surveys:

1. Avoid ambiguity
2. Avoid leading questions
3. Avoid lengthy surveys and very long responses
4. How will your initial questions influence answers to subsequent ones?
5. Think carefully about sampling techniques
6. Seek to achieve highest rate possible
7. Standardize administration procedures
8. Guarantee anonymity (or confidentiality at minimum)
9. Seek measures of reliability
10. Assess validity.

Primary Data Collection: Case Studies

• Opportunity to investigate issues deeply and descriptively.
• technically not a research method
• combination of various methods to form proper understanding of the proposed case:
  • gather data through qualitative methods including interviews and surveys
  • acquire secondary-sourced data e.g. essays and diaries for analysis
  • personally provided notes can be also utilized alongside official ones

Information Sources for Case Studies

• Direct observation from subjective evaluations
  • Single or group of observers
• Participant observation
  • Researcher participate in the setting like other under-study people and observe happenings from a closer perspective
• Conduct interviews with survey-type questions that are structured based on more open-ended question sets.
• Use various census and survey records, newspapers, letters, instruments, tools, etc.

Merit and Challenges of working with Case Studies

Pros/Merits:

• Combines the strengths of multiple research methods
• Consider research from various time frame: past, present and future
• Provide explanation about the changes and impacting factors that are not readily available

Cons/Challenges:

• Complex processes, time consuming and expensive
• No clear limit on when to stop collecting data
• The assumption taken may not always be realistic or data tested in that context
• Usually requires expert and trained conducting teams
• Over-interpretation and over-generalizing issues can happen (Taherdoost, 2021)

Primary Data Collection: Experimental

1. Laboratory/Controlled:
   • Highest control over study design and process,
   • gain precise and accurate data
2. Field:
   • real-life situation,
   • variables are manipulated still but
   • your control is lower than Scenario 1
3. Natural experiment:
   • no control over variables/environmental setting
   • very low reproducibility

Secondary data collection methods

Data gathered from published sources.

Challenges of Data Collection Process

1. Location of data collection

   • neutral location
   • participants to feel free to provide their responses

2. Literacy of Participants and Language of Questions

   • Design of questions is appropriate for the literacy level of participants
   • may require pilot tests to confirm (added costs)

3. Timing

   • Duration of test needs to be long enough to yield reasonable results
   • Short enough to maintain the engagement of the participants

4. Sensitivity of Data

   • Privacy of the Participants (Unique identifier)
   • Protecting personal information through promises and icebreakers and examples

Ethics in Observation Studies

When in doubt: Describe your observation plan to your instructor before starting.

Architecture/Urban Example: Observing Public Space Use

Research question: How do people use seating in Hong Kong’s privately-owned public spaces (POPS)?

This type of study informs urban design guidelines and policy.

Let’s Practice: Designing an Observation Study

Several years ago, a group of students at University of Central Arkansas conducted a study in which they observed the rate at which cars failed to stop at a campus stop sign and recorded whether the car had a student parking decal or a faculty/staff parking decal. This is obviously not fitting for Hong Kong context. Let’s perhaps picture a study of the rate of jay-walking at a traffic light instead - and record whether the pedestrian who crossed is a student/staff/tourist/local resident. Use what we have covered today to answer questions 1-7:

1. Which method of observation would be best? Justify your answer. Hint: back to participant/direct observations.
2. How would you schedule observations?
3. Define the categories of behavior that you would observe
4. Describe how you would optimize and measure the reliability of observations, including the use of independent observers and calculation of interobserver agreement.
5. Describe how you could use equipment for observation rather than human observers, what are the advantages and disadvantages?
6. Describe how you might use public records to answer the same research question. What might be some limitations of this approach
7. Describe how you might use a survey method to answer the same research question. What might be some limitations of this approach?

References

1. Frechtling, J. (2002). An overview of quantitative and qualitative data collection methods The 2002 user-friendly handbook for project evaluation (pp. 43-62).
2. Hox, J. J., & Boeije, H. R. (2005). Data collection, primary versus secondary Encyclopedia of social Measurement (Vol. 1): Elsevier.
3. Data collection challenges (2005).
4. Kabir, S. M. S. (2016). Methods Of Data Collection Basic Guidelines for Research: An Introductory Approach for All Disciplines (first ed., pp. 201-275).
5. Olsen, W. (2012). Data collecti on: Key debates and methods in social research (Vol. 1): Sage.
6. Pandey, P., & Pandey, M. M. (2015). Research Methodology: Tools and Techniques (Vol. 1). Romania: Bridge Center.
7. Rimando, M., Brace, A. M., Namageyo-Funa, A., Parr, T. L., Sealy, D.-A., Davis, T. L., . . . Christiana, R.W. (2015). Data collection challenges and recommendations for early career researchers. The Qualitative Report, 20 (12), 2025-2036.
8. Taherdoost, H. (2016a). How to design and create an effective survey/questionnaire; A step by step guide. International Journal of Academic Research in Management (IJARM), 5 (4), 37-41.
9. Taherdoost, H. (2016b). Measurement and scaling techniques in research methodology; survey/questionnaire development. International Journal of Academic Research in Management (IJARM),6 (1), 1-5.
10. Taherdoost, H. (2016c). Sampling methods in research methodology; how to choose a sampling technique for research. International Journal of Academic Research in Management (IJARM), 5 (2), 18-27.
11. Taherdoost, H. (2019). What is the best response scale for survey and questionnaire design; review of different lengths of rating scale/attitude scale/Likert scale. International Journal of Academic Research in Management (IJARM), 8 (1), 1-10.
12. Taherdoost, H. (2021). Handbook on Research Skills: The Essential Step-By-Step; Guide on How to Do a Research Project (Kindle ed.): Amazon.

Learning Objectives

By the end of this session, you will be able to:

1. Prepare for informational interviews through professional research
2. Craft effective outreach messages for professional networking
3. Apply thematic analysis to qualitative data (codes → themes)
4. Articulate the “so what?” of your findings

Common Patterns in Doc 0.1 Submissions

What typically works well:

• Clear articulation of research focus
• Adequate number of sources

Common areas for improvement:

• Formatting doesn’t match discipline conventions
• Synthesis missing — reads as summary, not analysis
• Theoretical framework underdeveloped
• Conclusions included (lit reviews don’t conclude — they set up your contribution)

Quick flash back on Document 1 submission

• Research-paper like
• ~80% of work needed to draft a research paper as it contains:
  • Abstract
  • Literature Review
  • Methodology
  • Hypothesis/Anticipated Conclusions
  • Relevant References
• How is this different from a research proposal?

Informational Interview/Coffee Chat

• What they are:
  • Professional/Casual Networking tools
    • Potential mentors, collaborators, future job opportunities
  • Sitting down with people who have started a career
  • Gather insights and build professional relationships
• Learning Opportunity
  • Someone’s firsthand experience
  • challenges & successes
  • career paths
  • realistic picture of what to expect in a particular field or role (Not an ask, but preferrably something that we can look at)

Setting Expectation

• Neither should bear agenda to secure job offer
• Learn and build connections
• Experience informational interview as a long-term investment
• Start building/expanding professional network
• The group project is a teaser of what you will likely experience going further into your career (regardless of discipline)

Quick Q&A on Information interviews

• Do you think you have a better idea of what these are by now?
• Any questions that you may have before we dive deeper into how to design/prepare for one?

Identify Person(s) of Intersts

• Who to reach out to?
  • Industry Leaders and Role Models
  • Alumni Network
  • Professional Associations
• Utilizing LinkedIn:
  • Advanced Sesarch Features
    • Filter professionals by industry, company, role and even location
  • LinkedIn Groups: Join groups of related interests.
• How to narrow scope of search?
  • Finding Common Grounds
    • Shared Interests
    • Recent Publications or Talks
  • Leverage Existing Connections: Academic/Industry
• Demo

Crafting Effective Cold Messages

Key Components:

1. Subject Line: Clear and engaging subject line that conveys the purpose of the message.
2. Personalized Greetings: Use the recipient’s name and personalized remark/compliment on recent accomplishments;
3. Introduction and Purpose: brief but effective.
4. Purpose Statement: seek advice, insights, experiences related to field of interest;
5. Make Connection: highlight commonalities shared with recipient
6. Express Genuine Interest
7. Specific Request: Call to Action with Flexibility
8. Politeness and Conciseness
9. Closing and Signature

Let’s checkout some examples. Give a probability that you will respond to a message that you receive on LinkedIn. (3 Examples)

Reaching Out:

Choosing the Right Platform:

• Platform Etiquette:
  • Norms and best practices for different platforms (LinkedIn, email, professional forums)
  • Initial outreach varies accordingly
• Professional Channels:
  • using professional channels for communication is important
  • make sure your profiles on these platforms are up-to-date and professional.
    • What do we mean by professional?

Timing and Frequency:

• Optimal Timing:
  • best times to send messages: avoiding weekends, late nights, and major holidays
  • the likelihood of a timely response improves accordingly.
• Follow-up Strategy:
  • follow-up strategy involves a waiting period (typically 1-2 weeks)
  • a polite follow-up message if there’s no initial response.

Reaching Out (Continued)

Managing Outreach Volume:

• Tracking System:
  • set up a simple tracking system (like a spreadsheet) to monitor whom you’ve contacted,
  • when, and any responses or follow-up actions needed.
• Balanced Approach:
  • ideally no mass messaging.
  • a balanced, targeted approach to outreach,
  • focusing on quality and personalization over quantity.

Dealing with Responses:

Positive Responses:

• Immediate Acknowledgment:
  • respond promptly to positive replies,
  • expressing gratitude for the willingness to engage and
  • proposing potential times for the chat or meeting.
• Preparation:
  • being well-prepared for the interaction,
  • with specific questions and goals in mind.

Negative or No Responses:

• Handling Rejection:
  • not everyone will have the time or interest to respond
  • it’s not a reflection of their worth or the quality of their outreach.
  • aka it might be very frustrating.
• Learning from Silence:
  • how would a lack of response offer learning opportunities?
  • should you change your outreach strategy or message content?

Know Your Client (KYC)

Do your homework so that you do not make mistakes.

Research Basics

• Background Check: reviewing LinkedIn profile, professional biographies, recent publications, news mentions.
  • Thorough professional research before outreach
• Understanding their work:
  • familiarize with the professional’s role/projects/contributions
  • understanding the context and significance of their work

Contextual Understanding

• Know the Industry: major trends, challenges and opportunities
• Company Culture: culture and values of where they work/associate with for talking points

Know Your Client (KYC) Cont’d

Preparing Questions: Structured/Semi-structured

• Informed Questions: Informed and Open-Ended, reflecting their research and curiosity about the professional
• Personalized Inquiry: avoid overly general or easily Google-able questions
• Object-Oriented: what do you want to achieve through the interviews
  • get first-hand critique on the discipline
  • know what day-to-day is like at the particular industry
• Set an agenda: what will happen if you don’t have one?

Activity: Research a Professional (10 min)

• Do background research on a potential interviewee you might be interested in reaching out to
• you should cover at least the following:
  • their professional role
  • list out companies they worked for
  • their career highlight/work most proud of
  • identify interviews/podcasts/presentation where they gave public opinion and based on those
  • come up with two questions that you think is relevant to ask

Meet Your Client

Showing Engagement:

• Active Engagement:
  • active listening,
  • showing interest through body language (in person or on video calls), and
  • asking follow-up questions based on the discussion.
• Note-taking:
  • taking brief notes during the conversation (with permission) to capture key points and advice,
  • demonstrates engagement and respect for the professional’s insights.

Respect and Professionalism:

• Time Management:
  • be mindful of the agreed-upon time for the conversation and
  • to avoid overextending unless the professional indicates they’re willing to continue.
• Confidentiality:
  • the need to respect any confidential or sensitive information shared during the conversation.

Meet Your Client (Continued)

Post-Interaction Reflection:

Reflective Practice:

• Review and Reflect:
  • review their notes and reflect on the conversation soon after,
  • identifying key learnings and any follow-up actions they might take.
• Feedback Loop:
  • consider what went well and what could be improved for future interactions
  • mindset of continuous improvement.

Mock Interview (If we have time ~ 15min)

• one-on-one practice: professional vs. student interviewer
• work together and determine who the professional will shadow (a real person)
• KYC on industry/persona individually
• student interviewer to come up with 3 questions as structured interview and one follow-up during the conversation
• student/professional to swap (if we have time)

Activities

1. Vote to review one literature review that was submitted (anonymized on mentimeter).
2. Selected literature reviewed by peers with the following criteria:

• Objective: Does the literature review sufficiently present its objective?
• Landscape: Do you think the literature review presented a clear theoretical framework of what are the relevant studies and state-of-the art research landscape?
• Caveat/Importance: Does the literature review present any clear indication that there is room for investigation of any existing caveats?
• Citations: Do you think the number of citations included is enough/adequate?
• Improvements: Anything that the author can do to strengthen the literature review?
• Challenge: On the topics that were discussed in the literature review, can you quickly leverage google scholar to find additional ones that investigates similar things?

3. We will review the remainder three of the five documents (so long as time permits) for this week.

Addendum: Checklist for successful literature review

1. Clarity and Coherence:

• Clear Objective: Define the specific goals and scope of the literature review.
• Logical Structure: Organize content in a logical manner, facilitating easy navigation and understanding.
• Language and Terminology: Use clear, concise language appropriate for an interdisciplinary audience, avoiding unnecessary jargon.

2. Problem Statement and Importance:

• Well-defined Issue: Clearly articulate the problem or research question the literature review addresses.
• Significance: Explain the importance and relevance of the problem within the context of design and computer science.
• Gap Identification: Highlight gaps or shortcomings in existing research that the literature review aims to address.

Addendum: Checklist for successful literature review (Cont’d)

3. Theoretical Framework:

• Conceptual Underpinnings: Present the theories or models that underlie the research area.
• Framework Integration: Demonstrate how the theoretical framework informs the literature review’s approach and analysis.
• Interdisciplinary Relevance: Ensure the framework is relevant and accessible to both design and computer science perspectives.

4. Comprehensive Coverage:

• Breadth and Depth: Cover a wide range of sources while diving deep into critical studies.
• Diverse Sources: Include academic journals, conference papers, books, and reputable online resources relevant to both fields.
• Timeliness: Ensure the inclusion of both foundational texts and recent research to reflect the current state of knowledge.

5. Critical Analysis:

• Comparative Analysis: Compare and contrast different studies, highlighting similarities and differences.
• Methodological Evaluation: Assess the methodologies used in key studies for their strengths and limitations.
• Theoretical Critique: Critically evaluate the theories discussed in the literature for their applicability and limitations in the interdisciplinary context.

Addendum: Checklist for successful literature review (Cont’d)

6. Synthesis:

• Thematic Organization: Synthesize literature thematically rather than summarizing each source individually.
• Insight Generation: Derive new insights or perspectives from the synthesis of the reviewed literature.
• Interdisciplinary Integration: Fuse insights from design and computer science to create a cohesive understanding.

7. Relevance to Research Question:

• Alignment: Ensure all reviewed literature contributes to answering the research question or addressing the problem statement.
• Application: Discuss how findings from the literature review apply to the specific intersection of design and computer science.

8. Source Evaluation:

• Credibility Assessment: Evaluate the credibility and reliability of each source.
• Bias and Perspective: Acknowledge potential biases in the literature and strive for a balanced perspective.

Addendum: Checklist for successful literature review (Cont’d)

9. Conclusions and Implications:

• Summary of Findings: Concisely summarize key findings and their implications for the research area.
• Future Research Directions: Identify areas where further research is needed, especially at the intersection of design and computer science.

10. Documentation and Referencing:

• Consistent Formatting: Adhere to a consistent citation style appropriate for the interdisciplinary audience.
• Accurate Citations: Ensure all sources are accurately cited within the text and in the reference list.

11. Reflection on Interdisciplinarity:

• Integration Challenges: Discuss any challenges encountered in integrating design and computer science literature and how they were addressed.
• Value of Interdisciplinary Approach: Reflect on how the interdisciplinary approach enriches the understanding of the topic.

Revisiting Report Submission

• Document 0: Research Statement (Extended Abstract)
• Document 0.1: Literature Review
• Document 0.2: Methodology & Data Needed/Collected
• Document 1: Research Proposal (Paper-like)

Check Moodle for current due dates.

Research Process in Flow Chart

Checklist for your final research proposal/paper submission

Elements that you’ve finalized:

• Research Problem Definition (Doc 0):
  • Articulated innovative challenge or aspect addressed.
  • Contextualized within industry/societal needs.
• Relevant Concepts and Theories (Doc 0.1):
  • Reviewed key theories underpinning the innovation area.
  • Included interdisciplinary approaches.
• Previous Research Findings (Doc 0.1):
  • Highlighted past innovations and research.
  • Noted successes and gaps for building upon/addressing.

Ready to be finalized:

• Hypothesis Formulation (Doc 0.2):
  • Developed clear, testable hypotheses predicting innovation outcomes.
• Research Design Formulation (Doc 0.2):
  • Outlined appropriate research methodologies/technologies to be used.
  • Discussed feasibility and potential impact of proposed research design.

Research Progress Check

Insofar, Your Design Should Provide:

• Evidence for Hypothesis:
  • Detailed data collection/analysis plan for hypothesis testing.
  • Consideration on data sources/appropriate technologies deployed in their collection.
• Significance:
  • Connected significance to societal/industry/academic trends.
  • Highlighted contribution to knowledge/innovation advancement.
• Implementation and Scalability:
  • Discuss real-world implementation pathways.
  • Address potential scalability barriers.
• Ethical Considerations and Sustainability:
  • Outline ethical considerations of the innovation.
  • Address sustainability of the proposed solution.

Final Step in Finalizing Research

As Finalizing the Proposal (Doc 1):

• Integration and Synthesis:
  • Ensured alignment of all proposal components with innovation/research goals.
• Stakeholder Engagement:
  • Identified key stakeholders: beneficiaries, participants, impacted communities.
  • Included stakeholder engagement plans: evidencing the effectiveness of proposed solution/design
• Consideration of Future Directions:
  • Speculated on future research directions to further the innovation.

The Analysis Gap

You’ve collected data. Now what?

• Interviews completed → pile of transcripts
• Survey closed → spreadsheet of responses
• Observations done → pages of field notes

The hard part isn’t collecting data. It’s making sense of it.

This section bridges that gap.

Thematic Analysis: The 5-Step Process

The most common approach for qualitative data:

Step 1: Familiarize — Read Without Agenda

First pass: Read everything without highlighting. Just absorb.

Second pass: Note anything that: - Surprises you - Repeats across participants - Contradicts what you expected - Feels emotionally charged

Anti-pattern: Jumping straight to coding. You’ll miss the forest for the trees.

Step 2: Code — Label Meaningful Chunks

A code = short label for a piece of data

Step 3: Search for Themes — Cluster Codes

Spread your codes out (physically or digitally). Look for patterns.

A theme is not a code. A theme is a pattern that says something meaningful about your research question.

Step 4: Review Themes — Stress Test

For each candidate theme, ask:

• Does it appear across multiple participants (not just one)?
• Does the data actually support it, or am I projecting?
• Is it distinct from other themes, or should they merge?
• Does it matter for my research question?

Kill themes that don’t hold up. 3-5 strong themes > 10 weak ones.

Step 5: Define & Name — Make It Stick

Each final theme needs:

1. A clear name (not jargon, not vague)
2. A 1-sentence definition
3. Representative quotes (2-3 per theme)

Worked Example: From 12 Interviews to 4 Themes

Research question: How do HKU students experience Instagram like-checking?

Raw data: 12 interviews, 45 pages of transcripts

Coding: 47 initial codes identified

Final themes:

1. Compulsive checking loop (10/12 participants)
2. Like thresholds as self-worth (8/12 participants)
3. Social comparison with peers (9/12 participants)
4. Temporary validation effect (7/12 participants)

Each theme has 2-3 supporting quotes and a clear definition.

Architecture/Urban Example: Analyzing Interviews About Public Space

Research question: How do Hong Kong residents perceive rule enforcement in privately-owned public spaces?

Raw data: 8 interviews with regular users of POPS in Central

Sample codes: - “Security told me to move” → enforcement encounter - “I didn’t know I couldn’t sit there” → unclear rules - “It doesn’t feel like public space” → ownership ambiguity - “I just avoid those places now” → behavioral adaptation

Emerging themes:

So what? Designers should consider how rule communication affects perceived publicness.

For Quantitative Data: From Numbers to Patterns

Key question: Is the pattern real, or just noise?

• p < 0.05 → statistically significant (probably not random)
• Effect size → practically significant (big enough to matter)

Triangulation: When Qual and Quant Meet

When they disagree: That’s interesting data, not a problem. Ask why.

The “So What?” Test

For every finding, ask: Why should anyone care?

If you can’t answer “so what?” — the finding isn’t ready.

Artifact Output: What You Leave With

Learning Objectives

By the end of this session, you will be able to:

1. Distinguish descriptive from analytical writing in your own drafts
2. Apply reverse outlining to identify structural gaps
3. Synthesize qualitative and quantitative findings using the Synthesis Canvas
4. Translate research findings into product hypotheses (Product Track)

Today’s Session Structure

1. Writing Workshop (40 min) — Descriptive vs. analytical writing + peer review
2. Synthesis Activity (25 min) — Working with the Synthesis Canvas
3. Research → Decision (20 min) — Product Track framework
4. Logistics (5 min) — Interview project reminder

Descriptive vs. Analytical Writing

Welcome to today’s session on distinguishing between descriptive and analytical writing. Let’s enhance our academic writing skills together!

Activity: Identify the Writing Style

We’ll go through several examples. For each, determine if it’s descriptive or analytical. After your response, we’ll reveal the correct answer and discuss.

Example 1: Question

“User engagement metrics have decreased by 20% over the past three months.”

Question:
Is this statement descriptive or analytical?

Example 1: Answer

Answer: Descriptive

Explanation:
The sentence reports a fact (a 20% drop) without delving into any reasons or implications behind the metric change.

Example 2: Question

“Maslow’s hierarchy of needs is a psychological theory that arranges human needs in a pyramid, with physiological needs at the base and self-actualization at the top.”

Question:
Is this statement descriptive or analytical?

Example 2: Answer

Answer: Descriptive

Explanation:
While it explains the pyramid structure, it does not analyze or critique the theory’s applicability or limitations across different cultures.

Example 3: Question

“The experiment demonstrated that plants exposed to sunlight grew faster than those kept in the shade.”

Question:
Is this statement descriptive or analytical?

Example 3: Answer

Answer: Descriptive

Explanation:
The statement provides an observation (faster growth with sunlight) without discussing the process or significance of the finding.

Example 4: Question

“In Shakespeare’s ‘Macbeth,’ Lady Macbeth’s manipulation of her husband’s actions serves as a commentary on the corrupting power of unchecked ambition and challenges traditional gender roles in Shakespearean society.”

Question:
Is this statement descriptive or analytical?

Example 4: Answer

Answer: Analytical

Explanation:
It not only identifies Lady Macbeth’s influence but also interprets her actions as a critique of societal norms, thus offering analytical insight.

Example 5: Question

“Survey results show that 60% of respondents prefer online shopping over in-store shopping.”

Question:
Is this statement descriptive or analytical?

Example 5: Answer

Answer: Descriptive

Explanation:
It presents a statistic without discussing why users might prefer online shopping or its implications for consumer behavior.

Example 6: Question

“While the interface layout includes a navigation bar, a content area, and a footer, this design might impede user engagement if it fails to prioritize the most critical user tasks, suggesting a need for a more dynamic, behavior-driven layout.”

Question:
Is this statement descriptive or analytical?

Example 6: Answer

Answer: Analytical

Explanation:
The statement goes beyond description by assessing the potential drawbacks of the layout and suggesting improvements based on user behavior.

Key Takeaways

• Descriptive vs. Analytical:
  • Descriptive writing states facts or observations.
    
  • Analytical writing explains the significance, reasoning, and implications behind those facts.
• Depth of Analysis:
  • Effective academic writing connects evidence with critical interpretation.
    
  • Always ask: “Why is this important?” and “What does this mean for the overall argument?”
• Practical Strategies for Improvement:
  • Peer Review: Leverage feedback from classmates to identify areas lacking analysis.
    
  • Reverse Outlining: Break down your text to pinpoint descriptive versus analytical sections.
• Continuous Revision:
  • Use these insights to refine your drafts, deepening your analysis and improving clarity.

Enhancing Your Analytical Writing

Objective:
- Develop your analytical writing skills by examining and revising your literature review or methodology sections.
- Engage in collaborative activities to practice critical analysis.

Today’s Activities:
1. Peer Review: In pairs, analyze each other’s writing using guided questions.
2. Reverse Outlining: Break down a text (yours or a provided sample) to identify strengths and gaps.

Stage 1: Peer Review Activity

Time: 25 minutes

Instructions:

1. Pair Up: Find a partner in the room.
2. Exchange Materials: Share your literature review or methodology section (or use the provided sample).
3. Guided Review: Use these questions as you read your partner’s work:
   • Thesis & Argument:
     • Is the thesis clear and arguable?
     • Does the argument show critical evaluation rather than just stating facts?
   • Use of Evidence:
     • Is evidence well integrated and analyzed?
     • Does the writer explain the significance of each piece of evidence?
   • Structure & Flow:
     • Are the ideas presented logically?
     • Do transitions connect the analysis throughout the paper?

Goal:
- Provide constructive feedback focused on deepening the analysis.

Stage 2: Reverse Outlining Exercise

Time: 25 minutes

Instructions:

1. Choose Your Text: Pick a section from the previously worked-upon example you looked at (4-5 paragraphs max).
2. Number Each Paragraph: Number the paragraphs in the margin for clarity.
3. Summarize Each Paragraph: In one sentence, capture the main idea or argument of each paragraph.
4. Evaluate the Outline:
   • Does each summary contribute to your overall argument?
     
   • Identify any parts that are merely descriptive rather than analytical.
5. Plan Revisions: Mark paragraphs that need more critical insight or better integration of evidence.

Goal:
- Reveal the structure and flow of your argument and present to each other. - Identify areas where you can add deeper analysis or clarity.

Reflection & Discussion

Discussion Questions:
- What new insights did you gain about your own writing from these activities?
- How can the feedback from your peer review inform your revisions?
- What specific changes will you make to improve the analytical depth of your paper?

Next Steps:
- Revise your draft based on the insights and feedback received. - Schedule a follow-up session if you need additional support on integrating analytical writing techniques.

Double-check your submission to make sure that it can be considered a followable instruction manual by the domain experts.

Using the Synthesis Canvas (25 min)

The challenge: You have findings from multiple sources. How do you bring them together?

• Qualitative themes from interviews
• Quantitative patterns from surveys
• Insights from literature

The tool: Synthesis Canvas (PDF)

Activity: Complete Your Synthesis Canvas

Step 1: What Data Do You Have? (3 min)

Fill in Section 1 of the canvas: - What qualitative data? (interviews, observations, open-ended responses) - What quantitative data? (survey scores, analytics, measurements) - How much of each?

Activity: Qualitative Synthesis (10 min)

Step 2: Pull Key Quotes

From your interviews/observations, identify 3-5 powerful quotes that capture key insights.

Step 3: Group Into Themes

Cluster related quotes. Name each theme. Note how many participants expressed it.

Activity: Triangulation Check (7 min)

Step 4: Do Qual and Quant Agree?

For each finding, check:

If they disagree: That’s interesting data. Write down why they might differ.

Activity: The “So What?” (5 min)

Step 5: Why Should Anyone Care?

For your top 2-3 findings, write one sentence explaining the implication:

If you can’t answer “so what?” — the finding isn’t ready for Doc 1.

Architecture/Urban Example: Synthesizing Mixed Methods Data

Research question: Does greenery in Hong Kong’s elevated walkways affect pedestrian stress levels?

Triangulation: All three methods converge — greenery correlates with lower stress and changed behavior.

So what? Urban designers should prioritize vegetation in elevated walkway design; the effect is measurable, not just aesthetic preference.

The Product Builder’s Challenge

You’ve done your research. You found the behavioral science. Now what?

Academic output: “Our findings suggest that social comparison mediates the relationship between like-checking frequency and anxiety (β = 0.38, p < .001).”

Product question: “Should we hide like counts? Will it actually help? How will we know?”

This section bridges that gap.

The Translation Problem

Your job: Turn “what we know” into “what we should build and why we expect it to work.”

The Research → Decision Canvas

Worked Example: Instagram Hidden Likes

Research finding: Social comparison theory (Festinger, 1954) + your qualitative data showing 9/12 participants check likes to compare with friends.

Case Study: Duolingo’s Streak Feature

The science: Loss aversion (Kahneman & Tversky, 1979) — people are more motivated to avoid losing something than to gain something of equal value.

The translation:

Result: Streaks became Duolingo’s core retention mechanism.

Case Study: Spotify Wrapped

The science: Self-presentation theory (Goffman, 1959) — people curate identity through what they share; nostalgia increases emotional engagement.

The translation:

Result: Wrapped became Spotify’s biggest annual marketing moment — built on behavioral science, not ad spend.

When Your Research Says “Don’t Build It”

Sometimes research reveals your hypothesis is wrong. That’s valuable.

Signals to reconsider:

• Literature shows the mechanism doesn’t work the way you thought
• Qualitative data shows users don’t actually have the problem you assumed
• Comparable products tried it and failed (with documented reasons)
• The effect size in existing research is too small to matter

What to do:

1. Document why you’re not building it (prevents revisiting the same dead end)
2. Look for adjacent opportunities the research revealed
3. Share with stakeholders — “We saved X months by researching first”

What Product Research Output Looks Like

Not a paper. A decision document.

Structure:

1. The Opportunity (1 paragraph): What problem are we solving? For whom?
2. What We Learned (1 page): Key findings from research — qual themes, quant patterns, literature insights
3. Our Hypothesis (1 paragraph): If we build X, then Y, because Z
4. Recommendation (1 paragraph): Build / Don’t build / Test first
5. Validation Plan (if building): How we’ll know if we’re right
6. Appendix: Raw data, quotes, methodology details

Length: 2-3 pages + appendix. Readable in 10 minutes.

The Screenshot-Worthy Slide

Example:

“We found that loss aversion (Kahneman & Tversky) predicts people work harder to protect what they have than to gain something new. Our interviews confirmed users hate ‘losing progress.’ This suggests that if we add a visible streak counter, users will practice daily to avoid breaking it — and we’ll measure success by 7-day retention rate.”

Artifact Output: What You Leave With

Group/Individual Interview Project

A foundational course in research methodology for design practice. We explore how rigorous inquiry informs innovation—from framing questions to gathering evidence and communicating findings.

Two Tracks, One Methodology:

Choose based on your interests. Same rigor. Same skills. Different outputs.

Key Resources

• Assessment Guide — Grading breakdown, document progression, submission guidelines
• Writing Samples — Examples of strong student work

Templates & Tools

Printable PDF toolkits — each includes a blank template on page 1 and a worked example on page 2.

Other Resources:

• Worksheet: Qualitative Methods — Quick-reference interview guide

Lectures & Materials

• Week 1: Introduction
  • Survey Data
• Week 2: Research Frameworks
• Week 3: Landscape Intelligence
• Week 4: Research Questions
• Week 5: Methods I
  • Worksheet
• Week 6: Methods II
  • Evidence and Narrative
  • Urban Green Space
• Week 7: Data Collection & Analysis
  • Part 1: Collection Methods
  • Part 2: KYC & Analysis
• Week 8: Research Paradigms & Methods
  • Peer Review Guide
• Week 9: Writing & Synthesis
  • Example Writeup
• Week 10: Data Visualization
  • Plotting Examples
  • Google Colab
  • Survey Data
  • Transcripts
  • Tabular Data
• Week 12: Presentations

Course Info

Office hours by appointment.